Search for: All records

Large Language Models (LLMs) have demonstrated unprecedented capabilities in code generation. However, there remains a limited understanding of code generation errors that LLMs can produce. To bridge the gap, we conducted an in-depth analysis of code generation errors across six representative LLMs on the HumanEval dataset. Specifically, we first employed open coding and thematic analysis to distill a comprehensive taxonomy of code generation errors. We analyzed two dimensions of error characteristics -- semantic characteristics and syntactic characteristics. Our analysis revealed that LLMs often made non-trivial, multi-line code generation errors in various locations and with various root causes. We further analyzed the correlation between these errors and task complexity as well as test pass rate. Our findings highlighted several challenges in locating and fixing code generation errors made by LLMs. In the end, we discussed several future directions to address these challenges. 

Abstract The relationships between adaptive evolution, phenotypic plasticity, and canalization remain incompletely understood. Theoretical and empirical studies have made conflicting arguments on whether adaptive evolution may enhance or oppose the plastic response. Gene regulatory traits offer excellent potential to study the relationship between plasticity and adaptation, and they can now be studied at the transcriptomic level. Here, we take advantage of three closely related pairs of natural populations of Drosophila melanogaster from contrasting thermal environments that reflect three separate instances of cold tolerance evolution. We measure the transcriptome-wide plasticity in gene expression levels and alternative splicing (intron usage) between warm and cold laboratory environments. We find that suspected adaptive changes in both gene expression and alternative splicing tend to neutralize the ancestral plastic response. Further, we investigate the hypothesis that adaptive evolution can lead to decanalization of selected gene regulatory traits. We find strong evidence that suspected adaptive gene expression (but not splicing) changes in cold-adapted populations are more vulnerable to the genetic perturbation of inbreeding than putatively neutral changes. We find some evidence that these patterns may reflect a loss of genetic canalization accompanying adaptation, although other processes including hitchhiking recessive deleterious variants may contribute as well. Our findings augment our understanding of genetic and environmental effects on gene regulation in the context of adaptive evolution. 

Abstract Changes in gene regulation at multiple levels may comprise an important share of the molecular changes underlying adaptive evolution in nature. However, few studies have assayed within- and between-population variation in gene regulatory traits at a transcriptomic scale, and therefore inferences about the characteristics of adaptive regulatory changes have been elusive. Here, we assess quantitative trait differentiation in gene expression levels and alternative splicing (intron usage) between three closely related pairs of natural populations of Drosophila melanogaster from contrasting thermal environments that reflect three separate instances of cold tolerance evolution. The cold-adapted populations were known to show population genetic evidence for parallel evolution at the SNP level, and here we find evidence for parallel expression evolution between them, with stronger parallelism at larval and adult stages than for pupae. We also implement a flexible method to estimate cis- vs trans-encoded contributions to expression or splicing differences at the adult stage. The apparent contributions of cis- vs trans-regulation to adaptive evolution vary substantially among population pairs. While two of three population pairs show a greater enrichment of cis-regulatory differences among adaptation candidates, trans-regulatory differences are more likely to be implicated in parallel expression changes between population pairs. Genes with significant cis-effects are enriched for signals of elevated genetic differentiation between cold- and warm-adapted populations, suggesting that they are potential targets of local adaptation. These findings expand our knowledge of adaptive gene regulatory evolution and our ability to make inferences about this important and widespread process.